A variational formula for risk-sensitive reward
نویسندگان
چکیده
We derive a variational formula for the optimal growth rate of reward in the infinite horizon risk-sensitive control problem for discrete time Markov decision processes with compact metric state and action spaces, extending a formula of Donsker and Varadhan for the Perron-Frobenius eigenvalue of a positive operator. This leads to a concave maximization formulation of the problem of determining this optimal growth rate.
منابع مشابه
A distribution-free risk-reward newsvendor model: Extending Scarf’s min-max order formula
Scarf’s min-max order formula for the distribution-free risk-neutral newsvendor problem is a classical result in the field of inventory management. The min-max order formula provides, in closed-form, the order quantity that maximizes the worst-case expected profit associated with the demand of a single product when only the mean and variance of the product’s demand distribution, rather than the...
متن کاملRisk-Sensitive Control and an Abstract Collatz–Wielandt Formula
The ‘value’ of infinite horizon risk-sensitive control is the principal eigenvalue of a certain positive operator. For the case of compact domain, Chang has built upon a nonlinear version of the Krein–Rutman theorem to give a ‘min–max’ characterization of this eigenvalue which may be viewed as a generalization of the classical Collatz–Wielandt formula for the Perron–Frobenius eigenvalue of a no...
متن کاملCOVARIANCE MATRIX OF MULTIVARIATE REWARD PROCESSES WITH NONLINEAR REWARD FUNCTIONS
Multivariate reward processes with reward functions of constant rates, defined on a semi-Markov process, first were studied by Masuda and Sumita, 1991. Reward processes with nonlinear reward functions were introduced in Soltani, 1996. In this work we study a multivariate process , , where are reward processes with nonlinear reward functions respectively. The Laplace transform of the covar...
متن کاملRobust Bounds on Risk-Sensitive Functionals via Rényi Divergence∗
We extend the duality between exponential integrals and relative entropy to a variational formula for exponential integrals involving the Rényi divergence. This formula characterizes the dependence of risk-sensitive functionals to perturbations in the underlying distribution. It also shows that perturbations of related quantities determined by tail behavior, such as probabilities of rare events...
متن کاملActor-Critic Algorithms for Risk-Sensitive MDPs
In many sequential decision-making problems we may want to manage risk by minimizing some measure of variability in rewards in addition to maximizing a standard criterion. Variance-related risk measures are among the most common risk-sensitive criteria in finance and operations research. However, optimizing many such criteria is known to be a hard problem. In this paper, we consider both discou...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- SIAM J. Control and Optimization
دوره 55 شماره
صفحات -
تاریخ انتشار 2017